Hypothesis Testing
Abraham Arbelaez
2024-03-26
What is hypothesis testing?
Hypothesis testing helps us making inferences about a population based on sample data. It involves evaluating two competing hypotheses about the population parameter of interest: the null hypothesis (\(H_o\)) and the alternative hypothesis (\(H_a\)). The goal is to assess whether there is enough evidence in the sample data to reject the null hypothesis in favor of the alternative hypothesis.
In other words, we are lawyers in a court and we want to prove whether the accused (our hypothesis) is guilty or not (hypothesis is true or not)! Innocent until proven guilty!
Hypothesis testing example: COVID-19
As we know, there was the COVID pandemic back in 2020. Back when the researchers were trying to create a cure, they came up with a research question: Does a COVID-19 vaccine significantly reduce the risk of infection among vaccinated individuals compared to unvaccinated individuals?
Once we have a research question, it is our duty to verify whether the claim that we are trying to test is true. That is why we have our null and alternative hypotheses:
Null Hypothesis
The null hypothesis represents the absence of an effect or no difference between vaccinated and unvaccinated individuals in terms of COVID-19 infection risk. In this case, the null hypothesis would be:
\(H_o\): The risk of COVID-19 infection is the same for vaccinated individuals as it is for unvaccinated individuals.
Alternative Hypothesis
The alternative hypothesis represents what the researcher is trying to find evidence for. In this case, the alternative hypothesis would be:
\(H_a\): The risk of COVID-19 infection is lower for vaccinated individuals compared to unvaccinated individuals.
In summary
\(H_o\): The risk of COVID-19 infection is the same for vaccinated individuals as it is for unvaccinated individuals.
\(H_a\): The risk of COVID-19 infection is lower for vaccinated individuals compared to unvaccinated individuals.
During the hypothesis testing process, researchers would collect data on COVID-19 infection rates among vaccinated and unvaccinated individuals. They would then analyze the data using appropriate statistical methods to determine whether there is enough evidence to reject the null hypothesis in favor of the alternative hypothesis. If the data provide strong evidence against the null hypothesis, the researchers may conclude that the COVID-19 vaccine is effective in reducing the risk of infection.
Following the same hypotheses tests that we are learning is how scientists were able to find the accuracy of the vaccine and allowed people to stop the spread through vaccination!
Test Statistics
In order to find evidence that our statement is correct, we need to use a test statistic. It is calculated from the sample and follows a known probability distribution under the assumption that the null hypothesis is true. Once we have our test statistic, we will have to find an appropriate significance level (denoted as \(\alpha\)), which is the probability of rejecting the null hypothesis when it is actually true.
The type of test statistic is based of the type of parameter that we are estimating. The scope of this class includes these population parameters:
Mean (\(\mu\))
Proportion (\(p\))
Variance (\(\sigma^2\))
Mean: t-test (\(\mu\))
We use the one-sample t-test to determine whether there is enough evidence to support a claim about the population mean based on sample data.
We first calculate our t test statistic by following the formula:
\[t = \frac{\bar{x}-\mu_0}{\frac{s}{\sqrt{n}}}\]
where:
\(\bar{x}\) is the sample mean
\(s\) is the sample standard deviation
\(n\) is the sample size
\(\mu_0\) is the specific value that we want to test
Once we have our \(t\) test statistic, we need to find our p-value. The p-value is the probability of observing a test statistic as extreme as, or more extreme than, the one calculated from the sample data, assuming that the null hypothesis is true.
For that, we use the t distribution (check out the cheat sheet).
Proportion (\(p\))
We use the test of proportion to determine whether there is enough evidence to support a claim about the population proportion based on sample data.
We first calculate our z test statistic by following the formula:
\[z = \frac{\hat{p}-p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}\]
where:
\(\hat{p}\) is the sample proportion
\(n\) is the sample size
\(p_0\) is the specific value that we want to test
Once we have our \(z\) test statistic, we need to find our p-value. For that, we use the normal distribution (check out the cheat sheet).
Variance (\(\sigma^2\))
To test variance, we must check whether our data behaves normally. We
use the Shapiro-Wilk
test with the function shapiro.test(). Once we know
that our data is normal, we need to calculate our test statistics by
following the formula:
\[\chi^2 = \frac{(n-1)s^2}{\sigma_0^2}\]
where:
\(s\) is the sample variance
\(n\) is the sample size
\(\sigma^2_0\) is the hypothesized population variance under the null hypothesis.
Once we have \(\chi^2\), we need to find our p-value. Since this has a chi-square distribution (\(\chi^2\)), check out the cheat sheet.
Significance Level and Decision Rule
We have our p-value, what now? This is when we decide whether our hypothesis is true or not (guilty or innocent).
If p-value < \(\alpha\), we reject the null hypothesis. This means, that we have enough evidence to reject our null hypothesis, in other words, our alternative hypothesis is true.
If p-value > \(\alpha\), we do not reject the null hypothesis/fail to reject the null hypothesis. This means, that we don’t have enough evidence to reject the null hypothesis. In other words, our alternative hypothesis is false.
Errors occur in hypothesis testing due to the inherent uncertainty involved in making statistical decisions based on sample data.
There are two types of error:
Type I Error: A Type I error occurs when the null hypothesis is incorrectly rejected, even though it is actually true. In other words, it’s the incorrect conclusion that there is a significant effect or difference when there is no true effect or difference.
Type II Error: A Type II error occurs when the null hypothesis is incorrectly not rejected, even though it is actually false. In other words, it’s the incorrect conclusion that there is no significant effect or difference when there is a true effect or difference.
We can see that the left image reflects that we reject the null hypothesis (affirming that the patient is pregnant), when in reality the patient isn’t pregnant.
The right image is the complete opposite, where we do not reject the null hypothesis (saying that the patient is not pregnant), when in reality the patient is pregnant.